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(S) Image coding method and image coding apparatus. 

(57) In high efficiency coding of motion image 
signal in transmission or recording of the mo- 
tion image signal, plural predict signals are 
determined by independently compensating a 
motion of reproduction signals of plural frames 
positioned before a frame to be coded. A linear 
combination of these predict signals is used as 
a predict signal. An error of the frame to be 
coded and the predict signal is coded. As a 
result, the distortion of the predict signal, that 
is, the predict error is reduced, and the coding 
efficiency is enhanced. 



CO 
(O 

T- 

<o 




0. 

Ill 



Jouve, 18. rue Saint-Denis, 75001 PARIS 



1 



EP 0 526 163 A2 



2 



The present invention relates to an image coding 
method for high efficiency coding of motion image sig- 
nals in transmission or recording of motion image sig- 
nals and an apparatus for performing the method. 

In the image coding apparatus, recently, along 
with the development of television telephone and tel- 
evision conference system, various high efficiency 
coding technologies have been realized. In particular, 
the coding technology making use of the inter-frame 
predict coding is widely used in the image coding ap- 
paratus. 

The Inter-frame predict coding system is to deter- 
mine a predict signal by predicting a frame to be en- 
coded from another frame, and encode a predict error 
which is a difference between the frame to be encod- 
ed and the predict signal. The predict signal is deter- 
mined in the following manners. (1) A reproduction 
signal of a frame before the intended frame to be en- 
coded is delivered from a memory of a local decoder, 
and it is compensated for motion to obtain a predict 
signal. (2) A reproduction signal of a frame before the 
intended frame to be encoded and a reproduction sig- 
nal of a frame after the intended frame are delivered 
from memories of a local decoder, and after they are 
compensated for motion, a mean value of them is de- 
termined as a predict signal. 

In the method of (2), in other words, a frame lo- 
cated between two reproduction signals apart from 
each other by two frames or more is determined by 
motion compensating frame interpolation, and this 
frame is used as the predict signal. One of the prior 
arts of such image coding apparatus is disclosed in 
EP90304841. The conventional image coding appa- 
ratus is explained below. 

The motion picture signals entering the image 
coding apparatus are subjected to two types of predict 
methods alternately on a frame by frame basis. The 
predict method is the manner of determining the pre- 
dict signal. The first predict method is to read the re- 
production signal two frames before from a frame 
memory of a local decoder, and compensate it for mo- 
tion to obtain a predict signal, and the second predict 
method is to determine an image signal by motion 
compensating frame interpolation from preceding and 
succeeding reproduction frames as a predict signal. 
Afterwards, a predict error signal is determined which 
is a difference between the frame to be encoded and 
the predict signal, and an encoded predict error signal 
is the output signal of the Image coding apparatus. 
The image coding apparatus, possessing a local de- 
coder, decodes the code, determines the reproduc- 
tion image, and stores it in a frame memory. 

In such constitution, however, a reproduction 
frame later In time is needed when determining the 
predict signal from the reproduction frames before 
and after the frame to be coded, that is, the frame later 
in time must have been already coded. To realize this, 
it is necessary to change the time sequence of the 



frames of the motion picture signals entering the cod- 
ing apparatus, and therefore the constitution of the 
coding apparatus Is complicated. 

It is hence a primary object of the invention to 

s raise the coding efficiency by reducing the distortion 
of predict signal, and to simplify the constitution of the 
coding apparatus. 

To achieve the above object, the invention pres- 
ents an image coding method comprising the steps of 

10 compensating a motion of reproduction signals of N 
(N being an integer of 2 or more) frames positioned 
before a frame to be coded to obtain first to N-th pre- 
dict signals, determining a linear combination of the 
first to N-th predict signals to obtain a predict signal, 

15 and coding a difference of the frame to be coded and 
the predict signal. 

The invention also presents an image coding ap- 
paratus comprising coding means for coding a differ- 
ence of a frame to be coded and a predict signal to 

20 produce a predict error code, decoding means for de- 
coding the predict error code to obtain a reproduction 
signal, memory means for storing the reproduction 
signal, motion compensating means for compensat- 
ing a motion of N (N being an integer of 2 or more) 

25 frames before the frame to be coded in the reproduc- 
tion signal stored In the memory means to obtain first 
to N-th predict signals, and linear combining means 
for determining a linear combination of the first to N- 
th predict signals to obtain the predict signal. 

30 Being thus composed, the invention is capable of 
determining N predict signals by compensating the 
motion of the reproduction signals of N frames pre- 
ceding the frame to be coded, and determining the 
predict signal by linear combining of these predict sig- 

35 nals, so that the distortion of predict signal can be re- 
duced. As a result, the predict error signal becomes 
small, and the coding efficiency is improved. 

Fig. 1 is a block diagram of an image coding ap- 
paratus in a first embodiment of the Invention. 

40 Fig. 2 is an explanatory diagram showing the re- 
lation between the frame of motion image signal to be 
coded, and a frame of reproduction signal used for de- 
termining a predict signal, in the image coding appa- 
ratus of the first embodiment of the invention. 

45 Fig. 3 is a timing chart of the image coding appa- 
ratus of the first embodiment of the invention. 

Fig. 4 is a block diagram of image coding appa- 
ratuses In second and third embodiments of the inven- 
tion. 

so Fig. 5 is an explanatory diagram showing the re- 
lation between a frame of moving image signal to be 
coded, and a frame of reproduction signal used for de- 
termining a predict signal, in the image coding appa- 
ratus of the third embodiment of the invention. 

55 Referring now to the drawings, an embodiment of 
an image coding apparatus of the invention is descri- 
bed in detail below. Fig. 1 is a block diagram of an im- 
age coding apparatus in a first embodiment of the in- 
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vention. In Fig. 1, numeral 1 denotes an input of the 
image coding apparatus, 2 is a predict error calcula- 
tor, 3 is a coder, 4 is a decoder, 5 is a predict signal 
calculator, 6 is a variable length coder, 7 is a buffer 
memory, 8 is a rate controller, and 9 is an output of the 
image coding apparatus. 

Numeral 301 is a discrete cosine transform (DOT) 
circuit, and 302 is a quantizer, which are combined to 
compose the coder 3. Numeral 401 is a dequantizer, 
and 402 is an inverse discrete cosine transform 
(IDCT) circuit, which are combined to compose the 
decoder 4. Numeral 502 is an adder, 503 is a first 
frame memory, 504 is a second frame memory, 505 
is a first motion compensator, 506 is a second motion 
compensator, 507 is an averager, and 508 is a motion 
estimator, which are combined to compose the predict 
signal calculator 5. 

Numeral 201 is a predict error signal produced by 
the predict error calculator 2, 303 is a predict error 
code produced by the coder 3, 403 is a reproduction 
predict error signal produced by the decoder 4, 501 is 
a predict signal produced by the predict signal calcu- 
lator 5, 5021 is a reproduction signal produced by the 
adder 502, 5051 is a first predict signal produced by 
the first motion compensator 505, 5061 is a second 
predict signal produced by the second motion com- 
pensator 506, 5081 is a first motion vector produced 
by the motion estimator 50B, 5082 is a second motion 
vector produced by the motion estimator 508, and 81 
Is the quantizing step size produced by the rate con- 
troller 8. 

In the thus composed image coding apparatus, 
the operation is described below while referring to Fig. 
1. The motion image signal to be coded is entered in 
the input 1 of the image coding apparatus, and Is fed 
into the predict error calculator 2 and the predict sig- 
nal calculator 5. The predict error calculator 2 deter- 
mines a difference of the motion image signal to be 
coded and the predict signal 501, and produces the 
result as the predict error signal 201 . The coder 3 re- 
ceives the predict error signal 201, processes it by 
OCT (discrete cosine transform) in the DCT circuit 
301, quantizes the obtained DCT coefficient value in 
the quantizer 302 according to the value of the quan- 
tizing step size 81, and produces the predict error 
code 303. The decoder 4 receives the predict error 
code 303, dequantizes it in the dequantizer 401 ac- 
cording to the value of the quantizing step size 81, 
processes the obtained DCT coefficient value by 
IDCT (inverse discrete cosine transform) in the IDCT 
circuit 402, and produces the reproduction predict er- 
ror signal 403. The variable length coder 6 receives 
the quantizing step size 81, predict error code 303, 
first motion vector 5081, and second motion vector 
5802, codes them in variable length, and writes the 
obtained data into the buffer memory 7. The data is 
read out of the buffer memory 7 at a specified rate, 
and produced Into the output 9 of the image coding ap- 



paratus. The rate controller 8 controls the value of the 
quantizing step size 81 so that the amount of data re- 
maining in the buffer memory 7 becomes a specified 
amount. 

5 The predict signal calculator 5 is a circuit for ob- 
taining the predict signal 501 . The adder 502 adds the 
reproduction predict error signal 403 and predict sig- 
nal 501, and produces the reproduction signal 5021. 
The reproduction signal 5021 is stored In the first 
10 frame memory 503 and second frame memory 504 al- 
ternately on a frame by frame basis. The first motion 
compensator 505 compensates the motion of the re- 
production signal read out from the first frame mem- 
ory by the first motion vector 5081 , and produces the 
is first predict signal 5051 . The second motion compen- 
sator 506 compensates the motion of the reproduction 
signal read out from the second frame memory 504 by 
the second motion vector 5082, and produces the 
second predict signal 5061. The averager 507 deter- 
mines an average of the first predict signal 5051 and 
second predict signal 5061, and produces the predict 
signal 501. The motion estimator 508 determines and 
produces a motion vector of the image signal to be 
coded. The motion estimator 508 determines the mo- 
tion vector between the image signal to be coded and 
the reproduction signal read out from the first frame 
memory 503, and produces the first motion vector 
5081. It also determines the motion vector between 
the image signal to be coded and the reproduction sig- 
nal read out from the second frame memory 504, and 
produces the second motion vector 5082. 

Explained next is the relation between the frame 
of the motion image signal to be coded and the frame 
of the reproduction signal used for determining the 
predict signal 501. Fig. 2 is an explanatory diagram 
showing the relation between the frame of motion im- 
age signal to be coded, and the frame of reproduction 
signal used for determining the predict signal 501, in 
which f(n-2) to f(n+3) denote the (n-2)-th (n+3)-th 
frames of image signals. Fig. 2 (a) shows a case of 
coding the frame f(n), in which the predict signal is de- 
termined by motion compensation of the frame f(n-2) 
and frame f(n-1) of reproduction signals. Fig. 2 (b) 
shows a case of coding the frame f(n+1), In which the 
predict signal Is determined by motion compensation 
of the frame f(n-1) and frame f(n) of reproduction sig- 
nals. Figs. 2 (c) and (d) show cases of coding the 
frame f(n+2) and frame f(n+3), respectively. 

The operation timing of the image coding appara- 
tus is described below while referring to Fig. 3. Fig. 3 
shows the timing chart of the image coding apparatus. 
In Fig. 3, (a) is an input signal of the image coding ap- 
paratus, (b) is the reproduction signal written into the 
first frame memory 503, (c) is the reproduction signal 
given to the first motion compensator 505 from the 
first frame memory 503, (d) is the reproduction signal 
written into the second frame memory 504, (e) is the 
reproduction signal given to the second motion com- 
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pensator 506 from the second frame memory 504, 
and (f) Is the predict signal 501. Besides, fr(n) is a 
frame of reproduction signal obtained by coding and 
decoding the frame f(n) of Input signal, and fp(N) is a 
frame of predict signal for coding the frame f(n) of in- 5 
put signal. 

The reproduction signal 5021 is written into the 
first frame memory 503 and second frame memory 
504 alternately on a frame by frame basis. When the 
frame f(n) is entered in the input 1 of the image coding 10 
apparatus as input signal, the frame fr(n-2) of repro- 
duction signal is read out from the first frame memory 
503, and the motion Is compensated by the first mo- 
tion compensator 505, so that the first predict signal 
5051 Is obtained. At the same time, the frame fr(n-1) 15 
of reproduction signal is read out from the second 
frame memory 504, and the motion is compensated 
in the second motion compensator 506, so that the 
second predict signal 5061 is obtained. The averager 
507 determines the average of the first pred let signal 20 
5051 and second predict signal 5061, and produces 
fp(n) as predict signal 501. 

In this way, since the average of the first and sec- 
ond predict signals Is used as the predict signal, the 
distortions contained in the first and second predict 25 
signals and are low In correlation are reduced by de- 
termining the average of the two predict signals. 
Thus, the distortion of predict signal is decreased, 
that is, the predict error is reduced, so that the coding 
efficiency is improved. 30 

In the first embodiment, meanwhile, the predict 
signal 501 was the average of two predict signals, that 
is, the first predict signal 5051 and second predict sig- 
nal 5052. But instead, by compensating the motion of 
N (N being an Integer of 2 or more) reproduction 35 
frames, the first to N-th predict signals can be deter- 
mined, and the predict signal 501 can be obtained by 
a linear combination thereof. 

In the first embodiment, moreover, the distance 
between the frame to be coded and the frame of the 40 
first or second predict signal is one or two frames, but 
this distance is not limitative. 

A second embodiment of the invention Is descri- 
bed below white referring to the accompanying draw- 
ings. 45 

Fig. 4 is a block diagram of an image coding ap- 
paratus in the second embodiment of the invention. In 
Fig. 4, numeral 509 is a predict mode decision circuit, 
510 is a predict signal selector, 5071 is a third predict 
signal produced by an averager 507, and 5091 is a so 
predict mode signal produced by the predict mode de- 
cision circuit 509. What is different from the first em- 
bodiment shown in Fig. 1 is the composition of the pre- 
dict signal calculator 5 and the input of the predict 
mode signal 5091 in the variable length coder 6. 55 

In the thus composed image coding apparatus, 
the operation is explained below while referring to Fig. 
4. 



The averager 507 determines an average of the 
first predict signal 5051 and second predict signal 
5061, and produces the third predict signal 5071. The 
predict mode decision circuit 509 Judges a smallest 
predict error out of the incoming first to third predict 
signals, and produces an index showing it as a predict 
mode signal 5091. The predictsignal selector 510 se- 
lects one out of the first predict signal 5051, second 
predict signal 5061 , and third predict signal 5071 , de- 
pending on the predict mode signal 5091 , and produc- 
es a selected signal as the predict signal 501 . The pre- 
dict signal 541 Is the one having the smallest predict 
error out of the first to third predict signals. Besides, 
the predict mode signal 5091 is entered in the variable 
length coder 6, and is coded in variable length togeth- 
er with the quantizing step size 81 , predict error code 
303, first motion vector 5081 , and second motion vec- 
tor 5082. 

Thus, according to the second embodiment, by 
installing the predict mode decision circuit and predict 
signal selector, the one having the smallest predict er- 
ror out of the first to third predict signals Is used as the 
predict signal, so that the distortion of the predict sig- 
nal can be reduced more as compared with the case 
of using the average of the first and second predict 
signals. 

A third embodiment is explained below. 

A block diagram of the third embodiment is the 
same as the second embodiment in Fig. 4. That is dif- 
ferent from the second embodiment is that the motion 
image signal entering the Input of the image coding 
apparatus is of interlaced scanning method. Fig. 5 is 
an explanatory diagram showing the relation between 
the frame of the motion image signal to be coded and 
the frame of reproduction signal used for determining 
the predict signal 501, in the image coding apparatus 
of the third embodiment of the invention, in the third 
embodiment since the image signal to be coded is of 
interlaced scanning method, the n-th frame is com- 
posed of field f (n, 1) and field f (n, 2). Fig. 5(a) shows 
a case of coding the field f(n, 1), In which the predict 
signal is determined by motion compensation of the 
field f(n-1 , 1) and field f(n-1 , 2) of reproduction signal. 
Fig. 5 (b) shows a case of coding the field (n, 2), in 
which the predictsignal is determined by motion com- 
pensation of the field f(n-1 , 1 ) and field f(n-1 , 2) of re- 
production signal. Figs. 5 (c) and (d) show cases of 
determining the field f(n+1, 1) and field f(n*1, 2). 

Thus, also when the motion image signal of inter- 
laced scanning method is coded by the coding appa- 
ratus in the third embodiment, the distortion of the pre- 
dict signal can be reduced in the same way as in the 
second embodiment Furthermore, since the first pre- 
dict signal and second predict signal are reproduction 
signals which are different from each other in the type, 
an efficient prediction is realized for a moving image 
signal. 

In the third embodiment, the two fields used for 
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determining the predict signal belong to one same 
field, but the two fields may also belong to different 
fields. 

In the third embodiment, moreover, the distance 
between the frame to be coded and the frame of the 
first or second predict signal is one frame, but this dis- 
tance is not limitative. 



Claims 

1 . An image coding method comprising the steps of: 

compensating a motion of reproduction 
signals of N (N being an integer of 2 or more) 
frames positioned before a frame to be coded to 
obtain first to N-th predict signals; 

determining a linear combination of the 
first to N-th predict signals to obtain a predict sig- 
nal; and 

coding a difference of the frame to be cod- 
ed and the predict signal. 

2. An image coding method comprising the steps of: 

compensating a motion of reproduction 
signals of two frames positioned before a frame 
to be coded to obtain a first predict signal and a 
second predict signal; 

determining an average of the first predict 
signal and the second predict signal to obtain a 
predict signal; and 

coding a difference of the frame to be cod- 
ed and the predict signal. 

3. An image coding method comprising the steps of: 

compensating a motion of reproduction 
signals of two frames positioned before a frame 
to be coded to obtain a first predict signal and a 
second predict signal; 

determining an average of the first predict 
signal and the second predict signal to obtain a 
third predict signal; 

determining a size of a predict error of the 
first, second and third predict signal In each block 
composed of a set of plural pixels; 

selecting one having a smallest predict er- 
ror out of the first, second and third predict signals 
to obtain a predict signal; and 

coding a difference of the frame to be cod- 
ed and the predict signal. 

4. An image coding method comprising the steps of: 

compensating a motion of reproduction 
signals of two frames positioned before a frame 
to be coded to obtain a first predict signal and a 
second predict signal; 

determining an average of the first predict 
signal and the second predict signal to obtain a 
third predict signal; 



selecting one out of two or more predict 
signals including the third predict signal to obtain 
a predict signal; and 

coding a difference of the predict signal 
5 and the frame to be coded. 

5. An image coding apparatus comprising: 

coding means for coding a difference of a 
frame to be coded and a predict signal to produce 
10 a predict error code; 

decoding means for decoding the predict 
error code to obtain a reproduction signal; 

memory means for storing the reproduc- 
tion signal; 

15 motion compensating means for compen- 

sating a motion of N (N being an integer of 2 or 
more) frames before the frame to be coded in the 
reproduction signal stored in the memory means 
to obtain first to N-th predict signals; and 

20 linear combining means for determining a 

linear combination of the first to N-th predict sig- 
nals to obtain the predict signal. 

6. An image coding apparatus comprising: 

25 coding means for coding a difference of a 

frame to be coded and a predict signal to produce 
a predict error code; 

decoding means for decoding the predict 
error code to obtain a reproduction signal; 

30 memory means for storing the reproduc- 

tion signal; 

motion compensating means for compen- 
sating a motion of N (N being an integer of 2 or 
more) frames before the frame to be coded in the 

35 reproduction signal stored in the memory means 
to obtain first to N-th predict signals; 

linear combining means for determining a 
linear combination of the first to the N-th predict 
signals to obtain a predict signal of a linear com- 

40 bination; and 

selector means for selecting one out of two 
or more predict signals including the predict sig- 
nal of linear combination to obtain the predict sig- 
nal. 

45 

7. An image coding apparatus comprising: 

predict error calculating means for produc- 
ing a difference of a frame to be coded and a pre- 
dict signal as a predict error signal; 
50 coding means for coding the predict error 

signal to produce a predict error code; 

decoding means for decoding the predict 
error code to obtain a reproduction signal; 

memory means for storing the reproduo- 
55 tion signal; 

compensating means for compensating a 
motion of two frames positioned before the frame 
to be coded In the reproduction signal stored in 
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the memory means to obtain a first predict signal 
and a second predict signal; 

averaging means for determining an aver- 
age of the first predict signal and the second pre- 
dict signal to obtain a third predict signal; and s 

predict error calculating means for deter- 
mining a size of a predict error of the first, second 
and third predict signals in every block which is a 
set of plural pixels; and 

means for selecting one having a smallest w 
predict error out of the first second and third pre- 
dict signals as the predict signal. 

8. An image coding apparatus comprising: 

predict error calculating means for produc- 1 5 
ing a difference of a frame to be coded and a pre- 
dict signal as a predict error signal; 

coding means for coding the predict error 
signal to produce a predict error code; 

decoding means for decoding the predict 20 
error code to obtain a reproduction signal; 

memory means for storing the reproduc- 
tion signal; 

compensating means for compensating a 
motion of two frames positioned before the frame 25 
to be coded in the reproduction signal stored in 
the memory means to obtain a first predict signal 
and a second predict signal; 

averaging means for determining an aver- 
age of the first predict signal and the second pre- 30 
diet signal to obtain a third predict signal; and 

selector means for selecting one out of two 
or more predict signals including the third predict 
signal to obtain the predict signal. 

35 

9. An image coding apparatus for coding a motion 
image signal of interlaced scanning method, 
comprising: 

predict error calculating meansfor produc- 
ing a difference of a field to be coded and a pre- 40 
diet signal as a predict error signal; 

coding means for coding the predict error 
signal to produce a predict error code; 

decoding means for decoding the predict 
error code to obtain a reproduction signal; 45 

memory means for storing the reproduc- 
tion signal; 

motion compensating means for compen- 
sating a motion of a first field and a second field 
positioned before the f ield to be coded in the re- so 
production signal stored in the memory means to 
obtain a first predict signal and a second predict 
signal; 

averaging means for determining an aver- 
age of the first predict signal and the second pre- 55 
diet signal to obtain a third predict signal; and 

predict error calculating means for deter- 
mining a size of a predict error of the first, second 



and third predict signals in every block which is a 
set of plural pixels; and 

means for selecting one having a smallest 
predict error out of the first, second and third pre- 
dict signals as the predict signal. 

10. An image coding apparatus for coding a motion 
image signal of interlaced scanning method, 
comprising: 

predict error calculating means for produc- 
ing a difference of a field to be coded and a pre- 
dict signal as a predict error signal; 

coding means for coding the predict error 
signal to produce a predict error code; 

decoding means for decoding the predict 
error code to obtain a reproduction signal; 

memory means for storing the reproduc- 
tion signal; 

motion compensating means for compen- 
sating a motion of a first field and a second field 
positioned before the field to be coded in the re- 
production signal stored in the memory means to 
obtain a first predict signal and a second predict 
signal; 

averaging means for determining an aver- 
age of the first predict signal and the second pre- 
dict signal to obtain a third predict signal; and 

selector means for selecting one out of two 
or more predict signals including the third predict 
signal to obtain the predict signal. 
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